Synchronization of computer peripheral effects

ABSTRACT

Synchronization of effects across multiple devices using local, distributed control, and eliminating the need for central direction of the effects with the associated large bandwidth required. In one embodiment, each device stores data and instructions for creating its own effects. The devices periodically communicate with a host or each other to maintain synchronization, and compensate for any drift. This approach minimizes the use of bandwidth and battery power in the devices.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of Ser. No. 14/830,404, filed MayAug. 19, 2015, titled, “SYNCHRONIZATION OF COMPUTER PERIPHERAL EFFECTS,”which is hereby incorporated by reference in its entirety for allpurposes.

BACKGROUND OF THE INVENTION

The present invention relates to the synchronization of lighting andother effects on computer peripheral devices.

Many devices, such as keyboards and mice, have decorative or indicatorlighting. In particular, many input and control devices used for gaminghave such lighting and other effects, such as vibrations. An exampledescribing keyboard lighting effects is US Pub. No. 20150108903,

Logitech and other keyboard and peripheral makers have keyboards andother devices with cosmetic lighting effects. Razer and Corsair makekeyboards with lighting effects that are customizable and can besynchronized with lighting effects on other devices, such as a mouse. Itis believed that such devices use a personal computer as thesynchronization source, with the PC sending out commands to each deviceover a wired USB connection. The PC coordinates the synchronization.

BRIEF SUMMARY OF THE INVENTION

The present invention provides the synchronization of effects acrossmultiple devices using local, distributed control, and eliminating theneed for central direction of the effects with the associated largebandwidth required. In one embodiment, each device stores data andinstructions for creating its own effects. The devices periodicallycommunicate with a host or each other to maintain synchronization, andcompensate for any drift. This approach minimizes the use of bandwidthand battery power in the devices.

In one embodiment, each device initiates an identified effect inresponse to a start signal from a host. The device then periodicallyprovides an indication to the host of where it is in the effectsequence. The host sends a synchronization correction message to thedevice when it is out of synchronization by more than a predeterminedamount. The report can simply be a timer indicating the position in theeffect, which is synchronized with a timer in the host. In oneembodiment, the device report signal is sent only once, in the firsthalf, or near the beginning of an effect cycle, such as ¼ or ⅓ into theeffect.

In one embodiment, the user can select an effect, and select to have itsynchronized with other devices. Once synchronized, the devices staysynchronized without further user input, even if the user changes theeffects.

In one embodiment, the effect is displayed simultaneously on a hostcomputer, and is separately generated on the host computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of multiple synchronized devices according toan embodiment of the invention.

FIG. 2 is a timing diagram illustrating the drift and synchronizingbetween a host and multiple devices according to an embodiment of theinvention.

FIG. 3 is a diagram of the messaging for synchronizing between a hostand a device according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a block diagram of multiple synchronized devices according toan embodiment of the invention. A keyboard 102, mouse 104 and personalcomputer 106 are shown. Keyboard 102 has a lighting system 108, andmouse 104 has a lighting system 110. Additional devices with their ownlighting systems, or other effects to be synchronized, can be added. Forexample, a game pad or game controller can also have lighting or othereffects. Other input devices would include a headset, a joystick,touchpad, steering wheel, or any other computer or gaming input device.Alternately, effects systems can be placed in non-peripheral devices,such as a mousepad, a stand, or the frame or display of the host itself.

Keyboard 102 also has a vibrator 112 and a speaker 114 for providinghaptic and sound effects which can be synchronized as well. A controller116 controls the various effects systems using a program in programmemory 118 and data in a data memory 120. In one embodiment, programmemory 118 can be partially in firmware or ROM. A transceiver 122 allowsthe wireless downloading of effects, and the wireless reporting ofeffect timing.

Mouse 104 can similarly have a vibrator 124 and speaker 126. It also hasa controller 128, data memory 130, program memory 132, transceiver 134and antenna 136. The transceiver and antenna also allows for thedownloading of synchronization effects and programs and the reportingback to personal computer 106 or to keyboard 102 of the status of thevarious effects.

Personal computer 106 has its own transceiver 140. Alternately, a routeror hub 142 could be used to interconnect the various devices thatprovide the communication link. In addition, the router or hub or otherdevice could function as the host, as well as providing thecommunication link.

FIG. 2 is a timing diagram illustrating the drift and synchronizingbetween a host and the devices according to an embodiment of theinvention. FIG. 2 shows a host 202, a device A (204) and a device B(206). These could be, for example, the PC, keyboard and mouse ofFIG. 1. When activated, device 204 sends a report 208 to the hostindicating that it has arrived and is active. The host can then send acommand 210 to the device to start the effect on the device(alternately, the device can notify the host when it is starting theeffect). At the same time, the host will independently compute and runthe effect itself (or start a timer for the effect period) so that itcan monitor the appropriate progress in the effect on the devices. Thecomputed effect can also be displayed on the host. Alternately, the hostcan simply run a timer for the effect.

Device 204 then periodically reports the effect status as indicated bycommands 212, 214 and 216. In the example shown, the host determinesthat there is 1% drift at report 212, 2% drift at report 214 and 3%drift at report 216. In this particular example, the host is programmedto reestablish synchronization, and correct the drift, when the driftequals or exceeds 3%. Thus, the host sends a synchronization signal 218to device 204. Device 204 continues to send reports, such as a report220, which may indicate a 1% drift again.

In the example shown, a second device 206 is activated and sends adevice arrival notification 222 to the host. The host then starts thesame effect on both devices at the same time, using command 224 todevice 204 and command 226 to device 206 (alternately, the same commandcan be broadcast to both). This is followed by the continuous reportsfrom devices 204 and 206 as shown, with the host sending asynchronization command to device 204 and device 206 when the reportsindicate that a particular device is 3% or more out of synchronization.

As can be seen in FIG. 2, Device A and Device B are different such thatDevice A tends to drift away from the host at a faster rate than DeviceB. When the device is attached, the host initiates the effect on thedevice and remembers the time (relative to the host) when the effect wasstarted. The device periodically sends an event to indicate the positionof the device effect. The event is sent from the device at around ⅓ ofthe effect period. For example, if the period of the event is 12seconds, the event is sent 4 seconds after the beginning of the effect.The host checks the position of the effect and compares it to its localposition. Once the drift has exceeded a heuristically determinedpercentage (usually based on what is perceptible to the user). a requestis sent to the device to go “backwards” or “forwards” relative to thedrift amount. The figure uses an example of 3% drift. At the end of theeffect, the device slows down or jumps ahead appropriately such that itis closer in sync with the host. When Device B is attached, the effectis restarted on all the devices, and the clock is reset on the host.Synchronization between multiple devices is accomplished by syncing tothe same host.

In more detail, in one embodiment, after the first device is activated,the host reads the effect parameters of the device. The host (LGS) takesthe period of the effect as a reference. The host runs a timerLGSTIMER_REF with the same period. LGS sends SWControl to enable device1. LGS can initialize its own LGSTIMER_REF with the device1 effectperiod counter just received. It will thus avoid large corrections onthe device. At this point LGS and device1 should have both timer effectssynchronized. If the device 1 period_counter drifts from LGSTIMER_REF,LGS sends the drift value to device 1. The drift value is calculatedthis way:drift=Min[Period−abs(LGSTIMER_REF−DeviceCounter),abs(LGSTIMER_REF−DeviceCounter)]the sign of drift is:negative ifPeriod−abs(LGSTIMER_REF−DeviceCounter)>abs(LGSTIMER_REF−DeviceCounter)positive ifPeriod−abs(LGSTIMER_REF−DeviceCounter)<abs(LGSTIMER_REF−DeviceCounter)

Device 2 can be subsequently activated. LGS reads effect parameters ofdevice 2. LGS makes sure the effect parameters are the same as device1,if not LGS should write the same settings (at least in terms of effectperiod). LGS sends SWControl enable to device 2. LGS receivesdevice2_period_counter, compares it to LGSTIMER_REF and sends to device2 the drift value, so that device 2 will make the adjustment. Dependingon the drift value, during the first steps the effect on device 2 canstop or speed up for a moment. The drift comparison is repeated on eachdevice.

FIG. 3 is a diagram of the messaging for synchronizing between a hostand a device according to an embodiment of the invention. This diagramis an example of a sawtooth effect started from a host to the device.The host keeps the master clock of the synchronized event. Due to errorson the device, the effect plays longer on the device than the host. Thehost waits for 3% drift before asking the device to synchronize itselfto the host. When the request is sent by the host to the device, thedevice adjusts the effect to finish the period quicker by the driftamount, as shown by abbreviated sawtooth 302. When the next period ofthe device effect starts, it is back in sync with the host effect. Inthe example of FIG. 3, the report from the device is sent near thebeginning of each cycle, or sawtooth, of the effect.

In one embodiment, a user interface is provided on the PC or other hostdevice. The user can select synchronization once, and thereaftersynchronization is maintained for all effects without further input fromthe user. The synchronization is maintained when devices power off andon again, with the effects starting again. The user can change theeffects, such as the type of effect, color, etc., and thesynchronization is carried over and maintained. In one embodiment, theUser Interface allows the user to apply the same effect to all devices.In another embodiment, the user can change the effect on one device. Ifsynchronization has been selected, the new effect will then bepropagated to all the other devices automatically by the host.

Various methods can be used to accomplish the synchronization. Thedevice can simply jump to the timing position the host commands.Alternately, the rate of change of the effect can be sped up, or sloweddown, so there isn't an abrupt change as it comes back intosynchronization. The change can take place in a portion of the effectwhere it will be less noticeable to the user, such as at the end of theeffect cycle, or where it is the least bright.

Various other embodiments can be used. For longer effect cycles,multiple reports can be sent. Alternately, instead of periodicallyreporting, a device can send a single report, such as when ¼ or ⅓ of theeffect has passed. This may be used, for example, for shorter effects.This cuts down on the amount of signaling required, with littlenoticeable difference for shorter effects, or for devices that maintainsynchronization well. In one embodiment, the host can monitor how well adevice stays synchronized over time, and can download a program changeto the device to require reporting more or less often, depending onhistorical performance.

In another embodiment, the host need not have its own time, but canreceive effect start and report signals from the multiple devices,compare them, and send out a correction to one or both. For example, onedevice could be instructed to slow down, and another to speed up, tolessen the abruptness of the change on one device.

In another embodiment, the devices can communicate directly with eachother, rather than with a host. For example, a bigger device, such as akeyboard, may have more processing power than, for example, a mouse. Thekeyboard could then act both as a device and the host. Alternately, thehost could be a smartphone or any other device.

In one embodiment, the program download to the device is done usingWiFi, while the reports and synch commands use Bluetooth. In oneembodiment, the reports and commands are sent as packets interspersedwith transmitted data on a WiFi or other wireless channel, such as sounddata to and from a headset.

A variety of effects can be used. For example, for lighting effects,individual keys or buttons could be lit up with different colors, or thesame color. A wave or other pattern can periodically sweep through thebuttons, or the effects can be provided on or around the keys hit by theuser. The effects could also light up the whole housing, or itsperimeter. The keys could be grouped in zones with the same color oreffect. A glowing effect could simply increase and decrease theintensity. The lighting effects can be coordinated with sound effects,haptic effects or other effects, or those effects could be used insteadof lighting effects.

In an alternate embodiment, the host can periodically send asynchronization command to the device, rather than having the deviceperiodically report. However, the inventors have determined that lesspower is used with device reporting, since the device then needs to beconstantly listening, and each command from the host must be processed,using power, regardless of whether synchronization is needed.

In one embodiment, effects are synchronized even though the effects aredifferent. For example, one device may use RGB (Red, Green, Blue)colors, while another uses CMYK (Cyan, Magenta, Yellow, and Key[black]). The effects may also be different in brightness, or could becompletely different effects that are coordinated in timing to achieve adesired overall effect, like the coordinating of different instrumentsin an orchestra.

In one embodiment, basic or simple effects are embedded in firmware orROM. More complicated effects are programmed in software and downloadedto the device, and can operate independently of the firmware effects, orcan utilize the firmware effects while adding to them. Alternately, morecomplex effects can be imbedded in higher value products. In oneembodiment, effects are programmed in firmware, with parameters that canbe downloaded to vary the effect.

As will be understood by those of skill in the art, the presentinvention can be embodied in other forms. Accordingly, the foregoingdescription is illustrating, but not limiting, of the scope of theinvention which is set forth in the following claims.

What is claimed is:
 1. A non-transitory computer readable mediaincluding instructions that configure one or more processors of a hostto perform the steps of: providing a first peripheral effect to a firstperipheral device; providing a second peripheral effect to a secondperipheral device; processing a signal indicating the synchronized startof the first peripheral effect and the second peripheral effect;receiving a report from the first peripheral device on a current timingof the first peripheral effect; receiving a report from the secondperipheral device on a current timing of the second peripheral effect;comparing the current timing of the first peripheral effect and thecurrent timing of the second peripheral effect to a host timercorresponding to the synchronized start of the first and secondperipheral effects; determining if the timing of the first peripheraleffect has varied from the host timer by more than a first predeterminedamount; sending a first synchronization signal to the first peripheraldevice if the first peripheral effect has varied from the host timer bymore than the first predetermined amount; determining if the timing ofthe second peripheral effect has varied from the host timer by more thana second predetermined amount; and sending a second synchronizationsignal to the second peripheral device if the second peripheral effecthas varied from the host timer by more than the second predeterminedamount.
 2. The computer readable media of claim 1 further comprisinginstructions that configure the processor on the host to perform thesteps of: communicating between the host and the first peripheral devicewith a first direct communication channel between a host transceiver anda first peripheral transceiver, and communicating between the host andthe second peripheral device with a second direct communication channelbetween the host transceiver and a second peripheral transceiver.
 3. Thecomputer readable media of claim 2 wherein the first and second directcommunication channels are USB or Bluetooth® transmission channels. 4.The computer readable media of claim 1 further comprising instructionsthat configure the processor on the host to perform the steps of:transmitting by the host to the first and second peripheral devices asignal indicating the synchronized start of the first and secondperipheral effects.
 5. The computer readable media of claim 1 whereinthe peripheral effects are each one of a lighting effect, a sound effector a haptic effect.
 6. The computer readable media of claim 1 whereinthe first and second peripheral effects are cyclic, and furthercomprising instructions that configure the processor on the host toperform the steps of: sending the first and second reports once in eachcycle of the first and second peripheral effects, in a first half of theperipheral effect cycle.
 7. The computer readable media of claim 1further comprising instructions that configure the processor on the hostto perform the steps of: indicating with a peripheral device timer aposition where it is in the peripheral effect; and tracking the positionin the peripheral effect with a host timer.
 8. The computer readablemedia of claim 1 further comprising instructions that configure theprocessor on the host to perform the steps of: providing a userinterface on the host allowing a user to change the peripheral effects;and maintaining the peripheral devices synchronized without further userinput after changes to the peripheral effects.
 9. The computer readablemedia of claim 1: wherein the first peripheral effect is a lightingeffect and the second peripheral effect is a sound effect.
 10. Thecomputer readable media of claim 1: wherein the first and secondsynchronization signals cause the first and second peripheral devices toreestablish synchronization by speeding up or slowing down the first orsecond peripheral effects.
 11. A method for controlling a host tosynchronize a first peripheral effect on a first peripheral device and asecond peripheral effect on a second peripheral device, comprising:providing the first peripheral effect to the first peripheral device;providing the second peripheral effect to the second peripheral device;processing a signal indicating the synchronized start of the firstperipheral effect and the second peripheral effect; receiving a reportfrom the first peripheral device on a current timing of the firstperipheral effect; receiving a report from the second peripheral deviceon a current timing of the second peripheral effect; comparing thecurrent timing of the first peripheral effect and the current timing ofthe second peripheral effect to a host timer corresponding to thesynchronized start of the first and second peripheral effects;determining if the timing of the first peripheral effect has varied fromthe host timer by more than a first predetermined amount; sending afirst synchronization signal to the first peripheral device if the firstperipheral effect has varied from the host timer by more than the firstpredetermined amount; determining if the timing of a second peripheraleffect has varied from the host timer by more than a secondpredetermined amount; and sending a second synchronization signal to asecond peripheral device if the second peripheral effect has varied fromthe host timer by more than the second predetermined amount.
 12. Themethod of claim 11 further comprising: communicating between the hostand the first peripheral device with a first direct communicationchannel between a host transceiver and a first peripheral transceiver,and communicating between the host and the second peripheral device witha second direct communication channel between the host transceiver and asecond peripheral transceiver.
 13. The method of claim 11 wherein thefirst and second direct communication channels are USB or Bluetooth®transmission channels.
 14. The method of claim 11 further comprising:transmitting by the host to the first and second peripheral devices asignal indicating the synchronized start of the first and secondperipheral effects.
 15. The method of claim 11 wherein the peripheraleffects are each one of a lighting effect, a sound effect or a hapticeffect.
 16. The method of claim 11: wherein the first peripheral effectis a lighting effect and the second peripheral effect is a sound effect.17. A system for synchronizing peripheral effects among multipleperipheral devices, comprising: a first peripheral device; a firstperipheral effect subsystem in the first peripheral device programmed toprovide a first peripheral effect over a period of time and to provide aseries of first timing reports on the current timing of the firstperipheral effect, wherein the first peripheral effect is one of alighting effect, a sound effect and a haptic effect; a first peripheraldevice data memory mounted in the first peripheral device and containingperipheral effect data usable to generate the first peripheral effect; atransceiver coupled to the first peripheral device and configured toreceive the first timing reports from the first peripheral effectsubsystem and transmit the reports wirelessly; the first peripheraleffect subsystem being programmed to receive a first synchronizationsignal and to modify the timing of the first peripheral effect inaccordance with the first synchronization signal; non-transitorycomputer readable media including instructions that configure aprocessor on a host to perform the steps of processing a signalindicating the synchronized start of the first peripheral effect and asecond peripheral effect; receiving the first report and a second reportand comparing them to a host timer corresponding to the synchronizedstart of the first and second peripheral effects; determining if thetiming of the first peripheral effect has varied from the host timer bymore than a first predetermined amount; sending the firstsynchronization signal to the first peripheral device if the firstperipheral effect has varied from the host timer by more than the firstpredetermined amount; determining if the timing of a second peripheraleffect has varied from the host timer by more than a secondpredetermined amount; and sending the second synchronization signal to asecond peripheral device if the second peripheral effect has varied fromthe host timer by more than the second predetermined amount.
 18. Thesystem of claim 17 wherein the non-transitory computer readable mediafurther includes instructions that configure the processor on the hostto perform the steps of: communicating between the host and the firstperipheral device with a first direct communication channel between ahost transceiver and a first peripheral transceiver, and communicatingbetween the host and the second peripheral device with a second directcommunication channel between the host transceiver and a secondperipheral transceiver.
 19. The system of claim 18 wherein the first andsecond direct communication channels are USB or Bluetooth® transmissionchannels.
 20. The system of claim 17 wherein the peripheral effects areeach one of a lighting effect, a sound effect or a haptic effect.